Skip to content

Apply rerun_with_latest_version to TriggerDagRunOperator reruns#67273

Open
nathadfield wants to merge 1 commit into
apache:mainfrom
king:apply-rerun-with-latest-version-to-trigger-dagrun-resets
Open

Apply rerun_with_latest_version to TriggerDagRunOperator reruns#67273
nathadfield wants to merge 1 commit into
apache:mainfrom
king:apply-rerun-with-latest-version-to-trigger-dagrun-resets

Conversation

@nathadfield
Copy link
Copy Markdown
Contributor

@nathadfield nathadfield commented May 21, 2026

Motivation

PR #63884 introduced a 4-tier rerun_with_latest_version config hierarchy (explicit > DAG-level > global config > fallback) controlling whether clearing/rerunning a Dag run uses the latest bundle version. That hierarchy is wired into the UI clear dialogs, the public clear/backfill API endpoints, and the airflow backfill create CLI command — but not into TriggerDagRunOperator's reset_dag_run=True path.

When TriggerDagRunOperator(reset_dag_run=True) is invoked and the target Dag run already exists, the task SDK trigger client gets a 409 and falls back to client.clear(dag_id, run_id). That hits POST /execution/dag-runs/{dag_id}/{run_id}/clear on the execution API, which previously called dag.clear(run_id=run_id) with no run_on_latest_version argument. As a result, the global and DAG-level rerun_with_latest_version config had no effect on operator-driven reruns.

What this PR does

Wires the existing resolve_run_on_latest_version resolver into the execution-API clear_dag_run endpoint, so operator-driven reruns honor the same hierarchy as UI-, API-, and CLI-driven reruns.

resolved_run_on_latest = resolve_run_on_latest_version(None, dag_id, session)
dag.clear(run_id=run_id, run_on_latest_version=resolved_run_on_latest)

None is passed as the explicit override because the operator does not (intentionally) accept a per-call override — the DAG-level toggle and global config from #63884 are already sufficient, and adding an operator parameter that accepts arbitrary versions would create a security gap (a git bundle could be made to run a commit that was never on the tracking branch).

Brand-new Dag runs from TriggerDagRunOperator are unaffected: they already use the latest bundle version because _create_orm_dagrun reads DagModel.bundle_version at trigger time. Only the rerun/clear path needed wiring.

No payload/schema change; no client change; no Cadwyn version bump.

Test plan

  • New unit test test_dag_run_clear_invokes_resolver verifies the resolver is invoked with None as the explicit override.
  • Existing TestDagRunClear tests still pass unchanged.
  • prek run --from-ref apache/main --stage pre-commit — all hooks pass (ruff, mypy-airflow-core, etc.).

closes: #60880


Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 4.7)

Generated-by: Claude Code (Opus 4.7) following the guidelines

@boring-cyborg boring-cyborg Bot added area:API Airflow's REST/HTTP API area:task-sdk labels May 21, 2026
@nathadfield nathadfield force-pushed the apply-rerun-with-latest-version-to-trigger-dagrun-resets branch 2 times, most recently from c5f2e4e to c17c7d6 Compare May 21, 2026 14:01
When `TriggerDagRunOperator(reset_dag_run=True)` is invoked and the
target Dag run already exists, the task SDK falls back to
`client.clear(dag_id, run_id)`, which hits
`POST /execution/dag-runs/{dag_id}/{run_id}/clear`. That endpoint
previously called `dag.clear(run_id=run_id)` with no
`run_on_latest_version`, so the global and DAG-level
`rerun_with_latest_version` config (added in apache#63884) had no effect on
operator-driven reruns.

The endpoint now resolves the value via the existing
`resolve_run_on_latest_version` hierarchy (explicit > DAG-level >
global config > fallback) and forwards it to `dag.clear()` — matching
the behavior already in place for UI, public API, and CLI clears.

closes: apache#60880
@nathadfield nathadfield force-pushed the apply-rerun-with-latest-version-to-trigger-dagrun-resets branch from c17c7d6 to 0900680 Compare May 22, 2026 07:43
@nathadfield nathadfield marked this pull request as ready for review May 22, 2026 09:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:API Airflow's REST/HTTP API area:task-sdk

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Apply rerun_with_latest_version to TriggerDagRunOperator reruns

1 participant